COST ESTIMATING RELATIONSHIPS FOR NAVAL 
SURFACE SHIP ELECTRONIC WARFARE EQUIPMENT 



Raymond Edward Moore 








COST ESTIMATING RELATIONSHIPS FOR NAVAL 
SURFACE SHIP ELECTRONIC WARFARE EQUIPMENT 



by 



Raymond Edward Moore III 



March 1975 



Thesis Advisor: J.K. Hartman 



Approved for public release; distribution unlimited. 

T 167559 



SECURITY CLASSIFICATION OF THIS PAGE (W.yn Dmtm Entered) 



REPORT DOCUMENTATION PAGE 


READ INSTRUCTIONS 
BEFORE COMPLETING FORM 


1. REPORT NUMBER 


2. GOVT ACCESSION NO. 


3. RECIPIENT'S CATALOG NUMBER 


«. TITLE (and Subtitle) 

Cost Estimating Relationships for Naval 
Surface Ship Electronic Warfare Equipment 


5. TYPE OF REPORT « PERIOD COVEREO 

Master’s Thesis 
March 1975 


6. PERFORMING ORG. REPORT NUMBER 


7. authorc*; 

Raymond Edward Moore III 


8. CONTRACT OR GRANT NUMBER^*; 


9. PERFORMING ORGANIZATION NAME AND ADDRESS 

Naval Postgraduate School 
Monterey, California 93940. 


10. PROGRAM ELEMENT. PROJECT, TASK 
AREA & WORK UNIT NUMBERS 


It. CONTROLLING OFFICE NAME AND ADDRESS 

Naval postgraduate School 
Monterey, California 93940 


12. REPORT DATE 

March 1975 


13. NUMBER OF PAGES 


M. MONITORING AGENCY NAME » AODRESS (II dill, font from Controlling Olllce ) 

Naval Postgraduate School 
Monterey, California 93940 


IS. SECURITY CLASS, (ol thl, report) 

Unclassified 


15a. DECL ASSIFICATIOn/ DOWNGRADING 
SCHEDULE 



16. DISTRIBUTION STATEMENT (of thl» Report) 



Approved for public release; distribution unlimited. 



17. DISTRIBUTION STATEMENT (of the ebeirect enured In Block 20, If different from Report) 



18. supplementary notes 



19. KEY WORDS (Continue on reveree tide If neceeeery and Identity by block number) 



20. ABSTRACT (Continue on reveree eldo If ntceeemry end Identity by block number) 

This study addresses the problem of estimating the development, 
procurement, and installation costs of surface ship electronic 
warfare equipment of the future. The Cost Estimating Relationships 
(CERs) were developed using the following factors: year of 

development, weight, volume, sensitivity, power output, gain, 
complexity and dummy variables for active equipment, equipment 
designed for large ships and one for those designed for small 
ships . 

DD t jan^73 1 473 EDITION OF 1 NOV 68 IS OBSOLETE 

(Page 1) S/N 0102-014- 660 1 | 



SECURITY CLASSIFICATION OF THIS PAGE (When Dete Entered) 



CbCU^TY CLASSIFICATION OF THIS page (Whmn D*tm Enffd, 



t 



i 



Cost estimates are made for three systems presently under 
development by Raytheon Company and Hughes Aircraft Company 
under a design-to-price program. 




DD Form 1473 (BACK) 
, 1 Jan 73 

S/N 0102-014-6G01 



SECURITY CLASSIFICATION OF -THIS P AGCfl/hAn Da/fi Ertfrmd) 



Cost Estimating Relationships for Naval 
Surface Ship Electronic Warfare Equipment 



by 



Raymond Edward Moore III 
Lieutenant, United States Navy 
B.S., United States Naval Academy, 1965 



Submitted in partial fulfillment of the 
requirements for the degree of 



MASTER OF SCIENCE IN OPERATIONS RESEARCH 



from the 

NAVAL POSTGRADUATE SCHOOL 
March 1975 



ABSTRACT 



This study addresses the problem of estimating the 
development, procurement, and installation costs of surface 
ship electronic warfare equipment of the future. The Cost 
Estimating Relationships (CERs) were developed using the 
following factors: year of development, weight, volume, 

sensitivity, power output, gain, complexity and dummy variables 
for active equipment, equipment designed for large ships 
and one for those designed for small ships. 

Cost estimates are made for three systems presently under 
development by Raytheon Company and Hughes Aircraft Company 
under a design-to-price program. 
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I . 



INTRODUCTION 



Traditionally cost estimates for military equipment have 
been derived through engineering techniques. These techniques 
are extremely time consuming and require detailed information 
about the proposed equipment. It has been found in recent 
years that estimates made using Cost Estimating Relationships 
(CERs) based on. equipment characteristics or parameters 
require less time. Some experts maintain that they are at 
least as accurate as the engineering estimates as long as care 
is taken in choosing the CER. 

An advantage of CERs is that since a proper CER is 
developed from historical data on similar equipment, it takes 
into account hidden costs which would be impossible, practically 
speaking, to separate out, and which are perhaps not consider- 
ed when making engineering estimates. 

Before one begins estimating costs by the parametric 
method, one must locate or develop a CER which adequately 
represents the equipment in question. 

The objective of this thesis is to develop such CERs to 
be used in estimating the costs for development, procurement, 
and installation of three design-to-price electronic warfare 
suites currently under development by Raytheon and Hughes 
for the United States Navy, and to estimate those costs for 
the proposed systems. 
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The CERs are developed from data presented in Ref. 1 
utilizing BIMED02R , a stepwise linear regression program 
developed by the University of California at Los Angeles (UCLA). 
This program considers all independent variables available 
and, at each step, includes in the equation the one which 
is most highly correlated with the dependent variable for 
the first step or the one with highest partial correlation 
with the previous step for all succeeding steps. This process 
may also remove a variable if that variable's F-value drops 
below the F-value to remove, designated for the particular 
regression, as more variables are added to the equation. 

More detailed information concerning BIMED02R is contained 
in Ref. 2. 

Section II of this thesis concerns the assumptions made 
about the data and explanations of the various parameters 
and variables available in the data base. 

The development of the CERs is presented in Section III 
along with an example demonstrating what can happen in linear 
regression if the independent variables in the equation are 
highly inter-correlated. Also included is an example of the 
use of a graphical method of selecting independent variables 
by Mallows found in Ref. 3. 

The CERs developed in Section III are used in Section IV 
to estimate the cost of the proposed systems and a comparison 
is made of those costs with the ones estimated in Ref. 1. 

The Appendix presents the data matrix, refers to its source 
and describes adjustments that were made to the original data. 



7 



II. DESCRIPTION OF THE MODEL 



The data for this thesis consists of the costs and 
characteristics of 21 electronic warfare (EW) systems developed 
from 1962 to 1974. These systems represents all of the major 
surface ship EW systems developed for the U.S. Navy during 
that time period. 



A. ASSUMPTIONS 

The following are assumptions made concerning the data 
used in this thesis. 

1. All of the figures in the data taken from Ref. 1 
are correct except as noted in the Appendix. It should 

be pointed out that this data differs greatly in some areas 
from the values found in Ref. 4, however, such differences 
can be explained by what aggregation of equipment one is 
referring to, such as, is one interested in the price of 
a whole suite or just the black box without antennas, etc. 
The data used in this thesis describes the total system 
for one ship. 

2. Costs for the individual equipments are independent 
of the costs of all other pieces of equipment. It is felt 
that this assumption is valid since cost of development of 
follow-on equipment which was a modification of existing 
equipment was adjusted where possible by Mitchell and 
Mullins for Ref. 1. Where such adjustment was found to be 
impossible, those data points were excluded from the data 
matrix. 



B. PARAMETERS AND VARIABLES 

The parameters and variables used in this thesis are 
described below. Dollars are measured in thousands of 1975 
dollars . 

1. Development Cost (DEVC) includes the cost of 
Research, Development, Test and Evaluation (RDT & E) of 
the equipment in question. 
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2. Procurement Cost (PRDC) is the cost of producing 
the first unit of the equipment. 

3. Installation Cost (INST)* is the average cost of 
installing the equipment on the surface ships for which 
it was designed. 

4. Year of Development (YR) is a measure of the number 
of years from the base year of 1969 to the year of develop- 
ment. This variable is negative for equipment developed 
prior to 1969, 

5. Weight (WT) is the weight in pounds of the equipment 

6. Volume (VOL) is the volume in cubic feet of the 
equipment . 

7. Sensitivity (SENS) is the receiver sensitivity of 
the equipment measured in decibels, All values are negative 

8. Power Output (KWO) is the power output of the 
active equipments measured in kilowatts. This variable 
is zero for passive equipment. 

9. Gain (GAIN) is the antenna gain measured in decibels 

10. Active (ACT) is a dummy variable with a value of 
one for active equipment and zero for passive ones. 

11. Complexity (CPLX) is a measure of complexity for 
the equipment, in this data base, ranging from one to three, 
one signifying relative simplicity and three extreme 
complexity. The source of this variable is discussed in 
the Appendix. 

12. Large (LG) is a dummy variable with a value of one 
for equipment designed for guided missile destroyers (DDG) 
and larger ships and zero otherwise. 

13. Small (SM) is a dummy variable with a value of one 
for equipment designed for destroyer (DD) and smaller ships 
and zero otherwise. For equipment designed for ships in 
both categories, both variables LG and SM are one. 



C. FORM OF THE MODELS 

Each model developed in this thesis is a linear regression 
equation containing one dependent cost variable, an intercept 
term and any number of the independent variables described 
above, numbers four through thirteen inclusive. 
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COST = a + a. YR + a. WT + ... + a, 0 SM 
o 4 o 13 

From examination of the residual plots at various stages 

during the development, there was no indication that polynomial 

or other transformations should be used. 
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III. DEVELOPMENT OF COST ESTIMATING RELATIONSHIPS 



With each CER, work was begun by examination of the 
correlation matrix and scatter plots of dependent versus each 
independent variable for those equipments for which data were 
available. Next, the data was fed into the BIMED02R stepwise 
linear regression program. Regression analysis was done with 
and without a constant term, that is without and with a zero 
intercept. In each case the fit was considerably better with 
a constant included; consequently, the zero intercept models 
were discarded. BIMED02R may be controlled as to the number 
of steps taken by designating a particular number of steps 
or by setting values for F--level for inclusion and deletion. 
Setting F-levels allows the variables to enter the model one 
at a time as long as their F-level to enter for the individual 
variable is higher than the level designated. If adding a 
new variable causes the F-level of a previously entered 
variable to decrease below the level set for deleting, that 
previously entered variable will be removed. In order to 
observe as much of the interaction as possible during the 
early analysis, no maximum number of steps was specified and 
F-levels were set at 0.01 for inclusion and 0.005 for deletion. 
When certain models appeared to be of interest, F-levels were 
increased to more realistic levels and later the maximum 
number of steps was designated in order to restrict the model. 
During this phase, residuals were listed and plotted against 
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the computer value for the dependent variable, and each of 
the independent variables. Each residual plot was examined 
for points that appeared to be outliers and for any indication 
of a need for any type of transformation. Eventually a best 
model was selected to be the CER for each of the three costs. 

In the statistics given after each CER, the F ratio is the 
test value for the hypothesis that the regression is not 
significent. If the F ratio is larger than the table value, 
the hypothesis is rejected and the regression is significant. 
The F value for each independent variable is used to test the 
hypothesis that the coefficient of the variable in question 
equals zero. For instance, if at a particular step there 
are three independent variables in the model, the F value 
for each variable is a measure of how much is lost by assuming 
that the model only includes the other two variables. The 
hypothesis is rejected if the table value is less than the 
F value. 

A . DEVELOPMENT COST 

Development costs were available for twelve (12) pieces 
of equipment. The correlation matrix for those equipments, 
listed below, was examined. 



DEVC 


YR 


WT 


VOL 


SENS 


KWO 




1.000 


0.384 

1.000 


0.685, 

0.283 

1.000 


0.772 
0.387 
0.870 v 
1.000 


-0.014 

0.162 

0.627 

0.468 


0.299 

0.443 

0.640 

-0.739 


DEVC 

YR 

WT 

VOL 



1.000 -0.692 SENS 

1.000 KWO 



12 



GAIN 


ACT 


CPLX 


LG 


SM 




0.213 


0.124 


0.846 


-0.214 


-0.108 


DEVC 


-0.251 


0.443 


0.631 


-0.328 


-0.426 


YR 


0.068 


0.511 


0.620 


-0.018 


-0.368 


WT 


-0.080 


-0.609 


-0.S21V 


0.326 


0.113 


VOL 


0.048 


-0.749 


-0.077 


0.075 


0.253 


SENS 


-0.172 


0.737 


0.491 


-0.507 


0.082 


KWO 


1.000 


0.067 


-0.003 


-0.014 


0.294 


GAIN 




1.000 


0.423 


-0.507 


-0.098 


ACT 






1.000 


-0.449 


-0.185 


CPLX 








1.000 


-0.577 


LG 










1.000 


SM 



Weight, volume, and complexity are highly correlated with 
development costs. However, since weight and complexity are 
also very highly correlated with volume, one must insure that 
if weight and complexity are included in the final model, 
volume is not and vice versa. 

When the data was processed by BIMED02R, the first four 
steps brought in CPLX, ACT, WT, and GAIN in that order. In 
each step the equation appeared acceptable; however, step 
four presented the best model. The CER at that step was: 



DEVC = -7474.87891 + .30165 WT + 142.60353 GAIN - 
3457.53076 ACT + 4757.61719 CPLX 
which had the following statistics: 



>2 

R‘ 



R = .9287 

,2 

(adjusted) is a monotonic increasing function with 



a final value of . 888 . 

Standard error of estimate = 1396.3611 

Mean value of the dependent variable = 3802.3 

F ratio = 22.782 

Table value 95 . 4 = 3.84 
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F value for each independent variable is: WT 8.8495 

GAIN 4.5523 
ACT 12.8525 
CPLX 35.1702 

Table value F, on .. = 3.46 

(. ,yo; l, »; 

This model is statistically appealing and intuitively 
reasonable. The negative sign on the constant is acceptable 
unless one is attempting to predict the development cost of 
a very light, low gain, simple piece of equipment. The 
equipment being considered in this thesis is heavy and complex 
Indications of trends in the data over time are that 
future equipments will have these same two characteristics. 
With the independent variables WT, GAIN, and CPLX as one 
would suspect, an increase in the value of any of the three 
will increase the development cost. The negative coefficient 
of the active dummy variable indicates that active equipment 
tend to be less expensive to develop for a given weight, gain, 
and complexity than passive equipment. This is understandable 
when one considers the different uses to which active and 
passive equipments are put. Active equipment is designed for 
a specific or a few specific types of manipulation of 
incoming signals over a small band of frequencies, then 
retransmission of these signals. Passive equipment on the 
other hand is used for the evaluation and identification of 
incoming signals over a very wide range of frequencies. The 
processing in this case must maintain the signal character- 
istics so that accurate identification may be made. 
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SYSTEM DEVELOPMENT COST 



in millions of 1975 dollars 



GRAPH 1 
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When residual plots were examined, neither outliers nor 
indication of a need for a transformation were found. \ Many 
transformations were tried however in an attempt to improve 
the model, but no improvement was noted. There are two 
computed values for development cost with negative signs. 

The equipment in question in each case is very simple and 
light, two hundred (200) pounds in the first case and 
twenty-five (25) pounds in the other, and thus in a range of 
little concern in this thesis. 





ACTUAL DEVC 


COMPUTED DEVC 


RESIDUAL 


SLQ19 


1886.0999 


148.4023 


1737.6975 


SLQ26 


3654.0000 


4915.4297 


-1261.4297 


SLQ28 


11694.0000 


11823.8711 


-129.8711 


SLQ30 


1707.0000 


1807.3008 


-100.3008 


SLQ30 2 


1600.0000 


1343.2773 


256.7227 


SLQ12A 


1754.0000 


2223.4141 


-469.4141 


SLQ17 


7256.0000 


7289.4141 


-33.4141 


SLQ12 


380.0000 


-90.0703 


470.0703 


WLR1 


3598.0000 


4770.0430 


-1172.0430 


WLR3 


45.0000 


-570.6680 


615.6680 


WLR8 


11694.0000 


9814.2344 


1879.7656 


WLR11 


360.0000 


2153.4727 


-1793.4727 



The standard error of estimate of this CER is very large, but 
is considered acceptable in light of the wide range of 
development costs, forty-five thousand dollars ($45,000) to 
eleven million six hundred ninety-four thousand dollars 
($11,694,000), and the small size of the data base. 

B . PROCUREMENT COST 

1 . Development Cost Excluded 

The data base for procurement cost contained all 
twenty-one (21) pieces of equipment. The correlation matrix, 
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listed below, showed that weight, volume, and complexity are 
very highly correlated with procurement costs and again both 
complexity and weight are highly correlated with volume. 



PROC 


YR 


WT 


VOL 


SENS 


KWO 




1.000 


0.542 


0.920 v 


0.920 


0.407 


0.593 


PROC 




1.000 


0.372 


0.492 


0.072 


0.465 


YR 






1.000 


0.878, 


0.513 


0.589 


WT 








1.000 


0.350 


0.642 


VOL 










1.000 


0.519 


SENS 












1.000 


KWO 



GAIN 


ACT 




CPLX 


LG 


SM 




0.286 


0.271 




0.735 ► 


-0.230 


-0.210 


PROC 


0.169 


0.151 




0.554 


-0.421 


-0.435 


YR 


0.195 


0.320 




0.602 


-0.125 


-0.281 


WT 


0.257 


0.361 




0.787 • 


-0.355 


-0.127 


VOL 


-0.144 


0.755 




0.074 


0.036 


-0.307 


SENS 


-0.013 


0.477 




0.349 


-0.508 


-0.207 


KWO 


1.000 


-0.122 




0.068 


-0.347 


-0.007 


GAIN 




1.000 




0.328 


-0.258 


-0.208 


ACT 








1.000 


-0.314 


-0.144 


CPLX 










1.000 


-0.085 


LG 












1.000 


SM 


When : 


regression 


analysis was 


performed 


on the ( 


data, the first 


four 


steps took 


in 


variables 


WT, CPLX, 


YR, and 


SM in that 


order 


. Any one 


of 


the steps 


furnishes 


a very ; 


satisfactory 



model, however step four results in the best. The fit in 
this case is exceptional and once again follows one's 



intuition . 



The CER was : 

PROC = -726.24341 + .22550 WT + 349.72095 CPLX + 
67.14648 YR + 324.85962 SM 



which had the following statistics 
R 2 = .959 
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R 

(adjusted) is a monotonic increasing function with a 
final value of .9487 

Standard error of estimate = 276.8726 

Mean value of the dependent variable = 956.82 

F ratio = 93.651 

Table value F, QC . . =2.96 

F value for each independent variable is: WT 152.7812 

CPLX 4 . 0917 

YR 10.3247 

SM 5.1110 

Table value F. __ =3.03 

( . 90 ; 1 , 1 /) 

As in the previous CER, the constant has a negative sign and 
is acceptable for the same reasons. The positive coefficients 
for WT and CPLX are as one would suspect since generally 
procurement costs rise as the values for those two indepen- 
dent variables increase. The positive coefficient for the 
dummy variable SM indicates that if a piece of equipment is 
being designed for use on destroyer and smaller ships, it is 
more expensive than those developed solely for larger ships. 
Those units which are designed for use on smaller ships are 
generally more compact and consequently more expensive to 
fabricate for a given weight. Variable YR also has a positive 
coefficient which can be explained by the fact that over the 
years included in the data base, prices for procurement have 
increased more than can be explained by weight and complexity. 

As can be seen from the table below, the computed first 
unit cost for three systems is negative, however since the 
three units concerned are of relatively low cost, they do not 
detract from the CER. It is also interesting to note that 
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-PREDICTED COST 



SYSTEM FIRST UNIT COST 
in millions of 1975 dollars 



GRAPH 2 



two of the three equipments with negative residuals are the 
same ones with negatives residuals for development cost. 





ACTUAL PROC 


COMPUTED PROC 


RESIDUAL 


ULQ5AX 


99.0800 


-45.6897 


144.7697 


ULQ6A 


128.2400 


107.6472 


20.5928 


ULQ6B 


130.6000 


346.9700 


-216.3700 


ULQ6C 


225.2800 


475.0002 


-249.7202 


SLQ19 1 


935.7998 


1468.3806 


-532.5808 


SLQ19 2 


588.0198 


717.5596 


-129.5398 


SLQ22 


959.0698 


930.2021 


28.8677 


SLQ23 


961.9900 


900.8877 


61.1023 


SLQ24 


359.6499 


500.4072 


-140.7573 


SLQ26 1 


2156.3799 


1896.6470 


259.7329 


SLQ26 2 


1120.7100 


930.7031 


190.0068 


SLQ28 


5226.1367 


5199.8320 


26.3047 


SLQ30 1 


1304.1199 


1551.7834 


-247.6636 


SLQ30 2 


821.6099 


773.4519 


48.1580 


SLQ12 


453.0898 


618.9177 


-165.8279 


SLQ17 


2725.0898 


2269.3391 


455.7507 


SLR12 


16.3000 


-275.1497 


291.4495 


WLR1 


79.3000 


336.6997 


-257.3997 


WLR3 


7.4600 


-448.9043 


456.3643 


WLR8 


1613.0198 


1572.1064 


40.9133 


WLR11 


175.0700 


259.1987 


-84.1287 


2. 


With Development Cost 


Included 





Although the author did not feel that using develop- 



ment cost, calculated by use of the CER in paragraph A, with 
the obvious uncertainty involved, was a good variable to be 



used in explaining the cost of procuring the first unit, he 
did feel that after a piece of equipment had been under 
development for a while a more accurate estimate of develop- 
ment cost would be available and could be used for that 
purpose. For that reason, regression analysis was performed 
using the twelve pieces of equipment for which development 



costs were known. 
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Although no acceptable CER was found, an interesting 
phenomenon occurred when the stepwise regression program was 
allowed to run with low F-level values and is presented here 



to demonstrate a problem concerned with high intercorrelations 

in a data base. The equations resulting from the various 

early steps were unacceptable due to very low F values for 

2 2 

the individual variables. The values for R and R (adjusted) 
increased continually and the standard error of estimate 
continued to decrease. Then suddenly at step nine, everything 
fit together and the F values of all the variables increased 
sharply. The following model was the result of that step: 

PROC = 2811.59351 + .05553 DEVC + .02824 WT + 3.81016 VOL + 
25.98703 SENS + 73,68718 KWO + 77.15399 GAIN - 
1667.51343 ACT - 1027.33765 LG - 1718.32324 SM 



which had the following statistics: 

R 2 = .9997 
r 2 

(adjusted) is a monotonic increasing function with a 



final value of .9973 



Standard error of estimate = 65.119 

Mean value of the dependent variable = 1291.67 

F ratio = 666.08 



Table value ^( 95 . g 3 ^ = 8.81 



F value for each independent variable: DEVC 

WT 

VOL 

SENS 

KWO 

GAIN 

ACT 

LG 

SM 



5.6128 

4.2697 

31.925 

67.1465 

102.7486 

130.7182 

62.6125 

41.8613 

133.5989 
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Table value F 



= 2,68 



(. 80 ; 1 , 3 ) 

At first glance, one would think this CER was exceptional; it 
does describe the data base very very well as shown by the 
statistics. However, if one examines the model itself and 
attempts to explain the coefficients, problems appear. 

One would have trouble explaining why an increase in the 
sensitivity of a piece of equipment that is a more negative 
number, would decrease the price, and why a piece of equipment 
designed for use on both a small and large ship with all other 
variables constant would-be $1.7 million cheaper per copy than 
one designed solely for large ships. Obviously the signs of 
the coefficients cannot be explained except by multicollin- 
earity, correlation between the independent variables in such 
a way that the different variables interact in the model to 
balance each other, and thus describe the particular data 
base extremely well. It must be remembered that the data 
set is small (12) and that nine parameters are used in the 
model. If one were sure that the intercorrelation experience 
in the past data would also occur in future equipment, this 
model could be used to good advantage; however, it is felt 
that such assurance is difficult to attain and should not 
be assumed. 

C. INSTALLATION COST ESTIMATING RELATIONSHIP 

Examining the data base, one finds that there is an 
installation cost available for nineteen of the equipments, 
however, since the SLQ 22, SLQ 23, and SLQ 24 were never 
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installed on any ship and since the estimated installation 
cost for each were higher than any other equipment in the data 
base, those three equipments were excluded from the data 
base used for this CER, 

Study of the correlation matrix, listed below, and scatter 
plots of the independent variables against installation cost 
offered only a preview of difficulties to come. 



INST 


YR 


WT 


VOL 


SENS 


KWO 




1.000 


0.203 


0.666 


0,618 


-0.338 


0.414 


INST 




1.000 


0.368 


0.449 


-0.317 


0.559 


YR 






1,000 


0.882 


-0.624 


0.615 


WT 








1.000 


-0.518 


0.712 


VOL 










1.000 


-0.575 


SENS 












1.000 


KWO 



GAIN 


ACT 


CPLX 


LG 


SM 




0.107 


0.044 


' 0.603 


-0.119 


-0.157 


INST 


0.176 


0.174 


0.439 


-0.347 


-0.515 


YR 


0.249 


0.296 


0.688 


-0.137 


-0.306 


WT 


0.331 


0.366 


0.829 


-0.389 


-0.137 


VOL 


0.097 


-0.805 


-0.497 


0.158 


0.289 


SENS 


-0.022 


0.408 


0.516 


-0.589 


-0.042 


KWO 


1.000 


-0.069 


0.139 


-0.271 


0.097 


GAIN 




1.000 


0.489 


-0.324 


0.022 


ACT 






1.000 


-0.262 


-0.262 


CPLX 








1.000 


-0.455 


LG 



1.000 SM 



No variable was highly correlated with installation cost, 
however, several independent variables were highly correlated 
with each other. 

When regression analysis was applied to the data, another 
interesting phenomenon occurred. First WT was taken into the 
model giving reasonable but poor statistics. During the 
next three steps, CPLX, ACT and SENS entered, in that order, 
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leaving an equation at the end of each step which was 
unacceptable due to low F values. On step five, WT was 
removed from the model and the statistics improved consider- 
ably. The CER at that step was; 

INST = 331.97876 + 254.31705 CPLX - 356,32886 ACT + 
8.42499 SENS 



which had the following statistics 

R 2 = .593 
R 2 

(adjusted) is a monotonic increasing function with a 
final value of .4913 

Standard error of estimate = 125.42 

Mean value of the dependent variable = 195.86 

F ratio = 5.828 

Table value F ( 9g . 3 _ 13) - 3.41 

F level of each independent variable: CPLX 9.7667 

ACT 6.7163 
SENS 4 . 3534 



Table value F 



(.90; 



1 , 



13) 



3.14 



The statistics for this CER are acceptable. As was indicated 
in the development CER, the passive equipment tends to be 
more expensive to install than active and increasing complexity 
of a unit, of course, increases the installation cost. A 
contradiction, however, does appear in the sensitivity 
coefficient. Since the sensitivity variable is measured in 
negative decibels, the more negative, the higher the sensi- 
tivity; intuition would require that the coefficient have a 
negative sign. In this case, increasing the sensitivity 
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decreases installation cost. Examining the correlation matrix 
again, it is found that sensitivity is highly negatively 
correlated with the active variable. This is easily under- 
stood when one realizes that passive equipment has high 
receiver sensitivity in order to maintain incoming signals at 
sufficient quality for evaluation. The assumption that this 
negative intercorrelation will remain true in future equipment 
is reasonable, consequently this CER is acceptable. 

Considerable investigation using various transformations 
of variables made no improvement over the above CER, so an 
attempt was made to find a better CER by a graphical method 
of selecting independent variables for inclusion by C.L Mallows 
described in Ref. 3. 

This method uses a statistic, C , which measures the sum 

P 

of the squared biases plus the squared random error. C^ is 
calculated by the formula: 

C = - (N - 2 ) 

P 2 v p' 



where 

RSS is the residual sum of squares 
2 

s is a good unbiased estimate of the variance of the 
random error inherent in the dependent variable 
N is the number of data points 
p is the number of terms in the equation 

2 

Reference 3 determines an estimator for s from an 
equation with all of the independent variables included. A 
forty-five degree straight line is drawn on a graph of number 
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of variables included versus C from the origin. This is 

P 

done since for p variables with negligible bias, = p. 

The distance from the horizontal axis to that line is random 

error and above it is bias error. At this point a C is 

P 

calculated for each possible equation containing one 

variable. The equation with the lowest contains the first 

variable to be included in the final model. Next C 's are 

P 

calculated for all models including the first variable and 

one of the remaining variables. The equation with the lowest 

C contains the first variable selected plus the second 

variable to be included in the final model. This process is 

continued adding one additional variable each step until all 

the variables are included. At this point there is one best 

equation for each step from which one must select the best 

model: in most cases it is the one with the lowest C value 
’ P 

regardless of whether the point is above or below the line. 
Normally one is willing to accept some bias in order to 
minimize variance. 

This method was attempted using the installation cost 

data with the hope that using to select independent 

variables would give better results than using correlation 

as used in stepwise regression analysis. It was felt that 

the model with all variables included did not explain the 

dependent variable very well and the statistic from that 

2 

regression confirmed it. That estimate of variance, s , as 
well as estimates of variance derived from several other 
regression equations were used in the formula for C , however, 
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none of the results improved upon the. model developed through 
stepwise regression. 

As a matter of fact, since the graphical method does not 

allow for removal of variables, as was used in the step 

regression analysis, the best model found by the C analysis 

P 

was one with WT and a constant only. 

One could use the long method, calculating for each 

of the 2^ possible combinations of variables, but it is 

not felt that the results would be worth the enormous amount 

of calcualtions required. Graph 4 shows the best results 

attained using Mallows method. The estimate of variance used 

is the minimum encountered during investigations of the 

various models for the installation CER. As can be seen, the 

process was only carried out to four variables. At that stage, 

two points (7, 11, 10) and (7, 11, 10, 13), found by stepwise 

regression but not by the C analysis, were below what should 

P 

have been the minimum C at each of steps three and four. It 

P 

is obvious that this method will not be of use on data with 
high intercorrelation until a way of removing variables is 
included . 
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IV. COST COMPUTATIONS AND ANALYSIS 



Presently under the Design-to-Price Electronic Warfare 
Program (DTP EW), Hughes Aircraft Company and Raytheon Company 
are each developing an electronic warfare system for the 
United States Navy. These systems are to have three stand- 
alone levels of operational capacity designated DTP-1, the 
least expensive, DTP-2, a moderately priced suite, and DPT-3, 
which includes DTP-1 and DTP-2. Present contracts require 
each company to develop and build two DTP-3 prototypes. 
Production decisions will then be made based on competitive 
prototype evaluation. Present plans are to build one hundred 
sixteen (116) DTP-1, one hundred-eighteen (118) DTP-2 and 
fifty-nine (59) DTP-3. 

Since the contract is a design-to-price type, prices have 
been specified, and the equipment is being developed to meet 
those prices. The cost targets are three hundred thousand 
dollars ($300,000), five hundred thousand dollars ($500,000) 
and one million four hundred thousand dollars ($1,400,000) 
respectively and are to include comulative average production, 
initial spares, and installation costs. Translating cumulative 
average production costs to first unit cost, the targets are 
six hundred eighteen thousand dollars ($618,000), one million 
thirty-two thousand dollars ($1,032,000), and two million 
two hundred thirty-five thousand dollars ($2,235,000). 
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All price estimates in this section were calculated using 
up-to-date parametric information received from PME 107X on 
19 February, 1975 as follows; 

RAYTHEON 





YR 


WT 


SENS 


GAIN 


ACT 


CPLX 


SM 


DTP-1 


6 


2458 


-55 


22 


0 


2 


1 


DTP-2 


6 


3932 


-55 


• 22 


0 


2 


1 


DTP- 3 


6 


8331 


-55 


22 


1 


3 


0 








HUGHES 










YR 


WT 


SENS 


GAIN 


ACT 


CPLX 


SM 


DTP-1 


6 


2095 


-63 


20 


0 


2 


1 


DTP -2 


6 


2933 


-63 


20 


0 


2 


1 


DTP- 3 


6 


6036 


-63 


14 


1 


3 


0 



All cost estimates are in thousands of 1975 dollars. Prediction 
interval (PI) are calculated for ninety-five percent confidence 
for each CER. 

A. DEVELOPMENT COST ESTIMATES 

The following development cost estimates were calculated 
using indicated CER in column one and two and are the program 
managers estimates as of 19 February 1975 in the last column. 





THESIS CER 
PI ( + 3220) 


REFERENCE 1 CER 
PI (+ 2406) 


PME 107 X 


RAYTHEON 


8990.77 


5055.95 


16,298 


HUGHES 


7157.65 


3663.47 


14,914 


Total 


16148.42 


8719.42 


31,212 



The CER from Ref* 1 is ; 

DEVC = 3.52335 + 1,92814 PROC 1 

PROC^ is the first unit price as determined by Ref. 1 
procurement CER number 1. The estimates using CERs are much 
lower prices than the program manager estimate. This is due 
to the fact that instead of just developing a system capable 
of one operational level as is the case in the data base, 

DTP-3 must be developed to operate at three levels. The 
calculations of development cost in Ref. 1 used a ninety 
percent cumulative learning for the development of two DTP-3 
systems. Since they are identical, logic would indicate that 
there would be no development cost for the second unit. It 
is interesting to note however, that by applying the learning 
curve, the CER from this thesis gives cost figures very close 
to those estimated by PME 107X as can be seen in the following 
table. The ninety percent learning was applied to columns 



one and two . 










THESIS CER 


REFERENCE 1 CER 


PME 107X 


RAYTHEON 


16,183.38 


9,100.7 


16,298 


HUGHES 


12,883.77 


6,594.24 


14,914 


TOTAL 


29,067.15 


15,694.94 


31,212 



B. FIRST UNIT PROCUREMENT COST ESTIMATES 

First unit procurement cost estimates were calculated 
using the CER as indicated in columns one and two. Column 
three contains the program's manager estimate of first unit 
procurement cost as of 19 February 1975. 
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THESIS CER 
PI(+ 584.2) 

RAYTHEON 


REF 1 CER 1 /CER 2 

PK± 1080/ ±1086 ) 


PME 107X 


DTP-1 


1255.22 


774.79/ 83g>51 


496.13 


DTP -2 


1587.55 


1238,49/ 12g736 


947.20 


DTP- 3 


2604.44 


2622.19/ 2623.93 


2005.32 


HUGHES 


DTP-1 


1173.36 


660.74/ 7 2g t 54 


513,41 


DTP- 2 


1362.33 


924 . 34 / 983 83 


880.24 


DTP- 3 


2086.92 


1900.44/ ig26 .63 


2261.10 



Ref. 1 used two CERs to estimate procurement cost, they are: 



PROC 1 = -1207.12 + .314561 WT - 18.7253 SENS + 6.62988 ERP 

and 

PR0C 2 = 92.6764 + .303836 WT 
ERP is effective radiated power. 

It would be impossible to explain the differences in 
prices between the three price estimates for each manufacturer's 
equipment in prices except to say that each was derived using 
a different estimator. In the case of Ref. 1, a smaller 
number of data point were used in determining each CER than 
this thesis. It is interesting to note that the CER developed 
in this thesis estimates consistantly $600,000 above PME 107X 
except for the Hughes DTP-3 for which the estimate is under 
PME 107X . 



33 



C, INSTALLATION COST ESTIMATE 



The cost estimates for installation costs in column one 



and two are 


calculated by the 


CER as indicated. 


Column three 


contains the 


program managers 


estimates as of 19 


February 1975 




THESIS CER 
PI (+270. 9) 


REF. 1 CER 
PI (+318 , 3 ) 


PME 107X 


RAYTHEON 


DTP-1 


377.24 


187.68 


43.15 


DTP- 2 


377.24 


239.00 


52.79 


DTP-3 


275.23 


359.38 


274.83 


HUGHES 


DTP-1 


309.83 


189.65 


56.1 


DTP- 2 


309.83 


231.38 


65.2 


DTP- 3 


207.81 


388.48 


145.2 



Ref. 1 used a CER based on first unit costs as estimated by 
PME 107X ; the CER is: 

INST = 131.24 + .113768 PROC 

In this case the large discrepency between the thesis estimate 
and PME 107X estimate is very easy to explain. First of all, 
the equipment is to be installed by the manufacturer while 
the ships are dockside at a Navy base rather than the usual 
practice of having it installed by shipyard personnel in Naval 
Shipyards. Secondly, as can be seen from the last section, 
the installation CER, although seemingly the best available 
from the data, is not particularly good. 

As can be seen from the cost estimates, the CER developed 
in this thesis predicts that DTP-3 installation cost will be 
less than either DTP-1, or DTP-2; however, this could not be 
correct since DTP-3 includes DTP-1 and DTP-2. This demonstrates 



34 



one of the problems that occur when one attempts to predict 
the cost of a system which is different than the equipment 
in the data base. Since the active coefficient has a 
negative sign with all other variables in the CER held 
constant, the active equipment DTP-3, installation cost 
estimate is less. 

D. SUMMARY AND CONCLUSIONS 

Cost Estimating Relationships are only as good as the 
data base from which they are derived. When applying them, 
one must be aware of whether the proposed system is within 
or is an extension beyond the data base. The larger the step 
beyond the data base, the more uncertain the results. 

In the case of the DTP EW program, many things are 
different than past equipments, consequently the need for 
more detailed costing is obvious. Installation costs, in 
particular, require a very detailed study since the whole 
idea of contractor installed equipment is new in EW equipment. 

This thesis estimated the costs development, first unit 
procurement and installation costs for the DTP EW system. 

The cost estimates for development cost, although lower than 
those predicted by the program manager, are acceptable when 
one understands the difference between development of 
equipment in the past and the DTP system. The procurement 
cost estimated by this thesis is consistantly $600,000 above 
the program manager estimates. The difference can only be 
explained by the fact that the prices were derived from 
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different estimators. In the case of installation cost 
estimates, the thesis CER estimates consistantly above program 
manager's estimates, however since the new equipment is to 
be contractor installed vice Naval Shipyard installed the 
difference would be expected. 
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APPENDIX: DATA BASE, SOURCE, AND ADJUSTMENTS 



The comprehensive data matrix of surface electronic warfare 
equipments, Table 1, was taken from Ref. 1, A considerable 
attempt was made to validate the particular numbers in the 
matrix, however, it was found that a validation of the cost 
figures, in most cases, would be impossible due to a lack of 
accurate documentation for the earlier equipments. The Air 
Force Electronic Countermeasures Directory, Ref. 4, was the 
only publication giving specific parametric information for 
all the systems, however, both the weight and volume figures 
differed considerably, in some cases as much as a factor of 
fourteen for WT, and ten for VOL. The data used in this 
thesis included the total package for a ship, while the 
figure given in the ECM Directory was only for the black box, 
antenna was not included nor were two units accounted for 
when two units were required per ship. 

Reference 1 stated that a 90% log linear cumulative 
average learning curve was assumed for EW equipment and used 
throughout that paper. It was decided that first unit procure- 
ment cost should be used, since different quantities of each 
equipment were purchased. Average unit costs and quantity 
were assumed to be correct and calculations were made to 
insure a ninety percent average learning curve was in fact 
used. These calculations indicated learning curves ranging 
from 80% to 90% with the great majority falling into the 
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84^86% range, The author of Ref, 1 charged with calculations 
could not explain the discrepency and stated that it could 
have been an error in his method of calculation. Consequently, 
first unit costs were recalculated using a 90% curve, average 
unit costs, and quantity purchased, except in the case of 
SLQ 22, SLQ 23, and SLQ 24 for which quantity purchased was 
zero; in that case first unit costs given in Ref. 1 were used. 
The author decided on 1969 vice 1900 as a base year and 
transformed the YR variable to conform. 

It was felt that a measure of complexity should be 
included as a possible independent variable. Mr. John P, 
Obrien, an Electronics Engineer in the Design-to-Price Office 
of Navy Electronic Systems Command PME 107X gave his expert 
opinion of each of the equipments in the data base as well 
as those three under development by Raytheon and Hughes. 

Each piece of equipment was given numerical va.lue of complexity 
ranging from 1, simple, through 2, moderately complex, to 3 
very complex. His evaluation was validated by other engineers 
in his office. 

Since the data base contained both passive and active 
equipments, a dummy variable with a value of one for active 
and zero for passive equipment was included. The only other 
possible difference between the equipment that it was felt 
could have some influence cost was the size ship the particular 
equipment was designed for. Two dummy variables were used 
to take this difference into account. The LG variable very 
simply was one if the equipment was designed for use on 
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Guided Missile Destroyers or larger ships and zero otherwise. 
Similarly the SM variable was one if the equipment was designed 
for destroyer or smaller ships. 

Table 2 is a listing of the data matrix as used in this 
thesis. Blanks in the matrix are data which is not available. 
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ELECTRONIC WARFARE EQUIPMENT DATA MATRIX 
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WLR-11 120 72 360 245.3 



ELECTRONIC WARFARE EQUIPMENT DATA MATRIX 
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